-
Open
-
Open
-
Open
-
Open
-
Open
-
Open
Autoregressive fragment-based diffusion for pocket-aware ligand design. (arXiv:2401.05370v1 [q-bio.BM])
In this work, we introduce AutoFragDiff, a fragment-based autoregressive
diffusion model for generating 3D molecular structures conditioned on target
protein structures. We employ geometric vector perceptrons to predict atom
types and spatial coordinates of new molecular fragments conditioned on
molecular scaffolds and protein pockets. Our approach improves the local
geometry of the resulting 3D molecules while maintaining high predicted binding
affinity to protein targets. The model can also perform scaffold extension from
user-provided starting molecular scaffold.
( 2
min )
TAnet: A New Temporal Attention Network for EEG-based Auditory Spatial Attention Decoding with a Short Decision Window. (arXiv:2401.05819v1 [eess.SP])
Auditory spatial attention detection (ASAD) is used to determine the
direction of a listener's attention to a speaker by analyzing her/his
electroencephalographic (EEG) signals. This study aimed to further improve the
performance of ASAD with a short decision window (i.e., <1 s) rather than with
long decision windows in previous studies. An end-to-end temporal attention
network (i.e., TAnet) was introduced in this work. TAnet employs a multi-head
attention (MHA) mechanism, which can more effectively capture the interactions
among time steps in collected EEG signals and efficiently assign corresponding
weights to those EEG time steps. Experiments demonstrated that, compared with
the CNN-based method and recent ASAD methods, TAnet provided improved decoding
performance in the KUL dataset, with decoding accuracies of 92.4% (decision
window 0.1 s), 94.9% (0.25 s), 95.1% (0.3 s), 95.4% (0.4 s), and 95.5% (0.5 s)
with short decision windows (i.e., <1 s). As a new ASAD model with a short
decision window, TAnet can potentially facilitate the design of EEG-controlled
intelligent hearing aids and sound recognition systems.
( 2
min )
Peridynamic Neural Operators: A Data-Driven Nonlocal Constitutive Model for Complex Material Responses. (arXiv:2401.06070v1 [cond-mat.mtrl-sci])
Neural operators, which can act as implicit solution operators of hidden
governing equations, have recently become popular tools for learning the
responses of complex real-world physical systems. Nevertheless, most neural
operator applications have thus far been data-driven and neglect the intrinsic
preservation of fundamental physical laws in data. In this work, we introduce a
novel integral neural operator architecture called the Peridynamic Neural
Operator (PNO) that learns a nonlocal constitutive law from data. This neural
operator provides a forward model in the form of state-based peridynamics, with
objectivity and momentum balance laws automatically guaranteed. As
applications, we demonstrate the expressivity and efficacy of our model in
learning complex material behaviors from both synthetic and experimental data
sets. We show that, owing to its ability to capture complex responses, our
learned neural operator achieves improved accuracy and efficiency compared to
baseline models that use predefined constitutive laws. Moreover, by preserving
the essential physical laws within the neural network architecture, the PNO is
robust in treating noisy data. The method shows generalizability to different
domain configurations, external loadings, and discretizations.
( 2
min )
Dynamic Indoor Fingerprinting Localization based on Few-Shot Meta-Learning with CSI Images. (arXiv:2401.05711v1 [cs.LG])
While fingerprinting localization is favored for its effectiveness, it is
hindered by high data acquisition costs and the inaccuracy of static
database-based estimates. Addressing these issues, this letter presents an
innovative indoor localization method using a data-efficient meta-learning
algorithm. This approach, grounded in the ``Learning to Learn'' paradigm of
meta-learning, utilizes historical localization tasks to improve adaptability
and learning efficiency in dynamic indoor environments. We introduce a
task-weighted loss to enhance knowledge transfer within this framework. Our
comprehensive experiments confirm the method's robustness and superiority over
current benchmarks, achieving a notable 23.13\% average gain in Mean Euclidean
Distance, particularly effective in scenarios with limited CSI data.
( 2
min )
Scaling up machine learning-based chemical plant simulation: A method for fine-tuning a model to induce stable fixed points. (arXiv:2307.13621v2 [cs.LG] UPDATED)
Idealized first-principles models of chemical plants can be inaccurate. An
alternative is to fit a Machine Learning (ML) model directly to plant sensor
data. We use a structured approach: Each unit within the plant gets represented
by one ML model. After fitting the models to the data, the models are connected
into a flowsheet-like directed graph. We find that for smaller plants, this
approach works well, but for larger plants, the complex dynamics arising from
large and nested cycles in the flowsheet lead to instabilities in the solver
during model initialization. We show that a high accuracy of the single-unit
models is not enough: The gradient can point in unexpected directions, which
prevents the solver from converging to the correct stationary state. To address
this problem, we present a way to fine-tune ML models such that initialization,
even with very simple solvers, becomes robust.
( 3
min )
New Online Communities: Graph Deep Learning on Anonymous Voting Networks to Identify Sybils in Polycentric Governance. (arXiv:2311.17929v4 [cs.LG] UPDATED)
This research examines the polycentric governance of digital assets in
blockchain-based Decentralized Autonomous Organizations (DAOs). It offers a
theoretical framework and addresses a critical challenge facing decentralized
governance by developing a method to identify sybils, or spurious identities.
The method uses graph deep learning techniques to identify sybil activity in a
DAO governance dataset (snapshot.org). Specifically, a Graph Convolutional
Neural Network (GCNN) learned voting behaviours and a fast k-means vector
clustering algorithm (FAISS) used the high dimensional embeddings to identify
similar nodes in a graph. The results reveal that deep learning can effectively
identify sybils, reducing the voting graph by 2-5%. This research underscores
the importance of sybil resistance in DAOs and offers a novel perspective on
decentralized governance, informing future policy, regulation, and governance
practices.
( 2
min )
Beyond Gradient and Priors in Privacy Attacks: Leveraging Pooler Layer Inputs of Language Models in Federated Learning. (arXiv:2312.05720v2 [cs.LG] UPDATED)
Federated learning (FL) emphasizes decentralized training by storing data
locally and sending only model updates, underlining user privacy. Recently, a
line of works on privacy attacks impairs user privacy by extracting sensitive
training text from language models in the context of FL. Yet, these attack
techniques face distinct hurdles: some work chiefly with limited batch sizes
(e.g., batch size of 1), and others are easily detectable. This paper
introduces an innovative approach that is challenging to detect, significantly
enhancing the recovery rate of text in various batch-size settings. Building on
fundamental gradient matching and domain prior knowledge, we enhance the attack
by recovering the input of the Pooler layer of language models, which enables
us to provide additional supervised signals at the feature level. Unlike
gradient data, these signals do not average across sentences and tokens,
thereby offering more nuanced and effective insights. We benchmark our method
using text classification tasks on datasets such as CoLA, SST-2, and Rotten
Tomatoes. Across different batch sizes and models, our approach consistently
outperforms previous state-of-the-art results.
( 2
min )
Adaptive Estimation of Random Vectors with Bandit Feedback: A mean-squared error viewpoint. (arXiv:2203.16810v3 [cs.LG] UPDATED)
We consider the problem of sequentially learning to estimate, in the mean
squared error (MSE) sense, a Gaussian $K$-vector of unknown covariance by
observing only $m < K$ of its entries in each round. We first establish a
concentration bound for MSE estimation. We then frame the estimation problem
with bandit feedback, and propose a variant of the successive elimination
algorithm. We also derive a minimax lower bound to understand the fundamental
limit on the sample complexity of this problem.
( 2
min )
Enhancing Blood Flow Assessment in Diffuse Correlation Spectroscopy: A Transfer Learning Approach with Noise Robustness Analysis. (arXiv:2401.05580v1 [cs.LG])
Diffuse correlation spectroscopy (DCS) is an emerging noninvasive technique
that measures the tissue blood flow, by using near-infrared coherent
point-source illumination to detect spectral changes. While machine learning
has demonstrated significant potential for measuring blood flow index (BFi), an
open question concerning the success of this approach pertains to its
robustness in scenarios involving deviations between datasets with varying
Signal-to-Noise Ratios (SNRs) originating from diverse clinical applications
and various setups. This study proposes a transfer learning approach, aims to
assess the influence of SNRs on the generalization ability of learned features,
and demonstrate the robustness for transfer learning. A synthetic dataset with
varying levels of added noise is utilized to simulate different SNRs. The
proposed network takes a 1x64 autocorrelation curve as input and generates BFi
and the correlation parameter beta. The proposed model demonstrates excellent
performance across different SNRs, exhibiting enhanced fitting accuracy,
particularly for low SNR datasets when compared with other fitting methods.
This highlights its potential for clinical diagnosis and treatment across
various scenarios under different clinical setups.
( 2
min )
Quantifying Marketing Performance at Channel-Partner Level by Using Marketing Mix Modeling (MMM) and Shapley Value Regression. (arXiv:2401.05653v1 [cs.LG])
This paper explores the application of Shapley Value Regression in dissecting
marketing performance at channel-partner level, complementing channel-level
Marketing Mix Modeling (MMM). Utilizing real-world data from the financial
services industry, we demonstrate the practicality of Shapley Value Regression
in evaluating individual partner contributions. Although structured in-field
testing along with cooperative game theory is most accurate, it can often be
highly complex and expensive to conduct. Shapley Value Regression is thus a
more feasible approach to disentangle the influence of each marketing partner
within a marketing channel. We also propose a simple method to derive adjusted
coefficients of Shapley Value Regression and compares it with alternative
approaches.
( 2
min )
TRLS: A Time Series Representation Learning Framework via Spectrogram for Medical Signal Processing. (arXiv:2401.05431v1 [eess.SP])
Representation learning frameworks in unlabeled time series have been
proposed for medical signal processing. Despite the numerous excellent
progresses have been made in previous works, we observe the representation
extracted for the time series still does not generalize well. In this paper, we
present a Time series (medical signal) Representation Learning framework via
Spectrogram (TRLS) to get more informative representations. We transform the
input time-domain medical signals into spectrograms and design a time-frequency
encoder named Time Frequency RNN (TFRNN) to capture more robust multi-scale
representations from the augmented spectrograms. Our TRLS takes spectrogram as
input with two types of different data augmentations and maximizes the
similarity between positive ones, which effectively circumvents the problem of
designing negative samples. Our evaluation of four real-world medical signal
datasets focusing on medical signal classification shows that TRLS is superior
to the existing frameworks.
( 2
min )
EMG subspace alignment and visualization for cross-subject hand gesture classification. (arXiv:2401.05386v1 [eess.SP])
Electromyograms (EMG)-based hand gesture recognition systems are a promising
technology for human/machine interfaces. However, one of their main limitations
is the long calibration time that is typically required to handle new users.
The paper discusses and analyses the challenge of cross-subject generalization
thanks to an original dataset containing the EMG signals of 14 human subjects
during hand gestures. The experimental results show that, though an accurate
generalization based on pooling multiple subjects is hardly achievable, it is
possible to improve the cross-subject estimation by identifying a robust
low-dimensional subspace for multiple subjects and aligning it to a target
subject. A visualization of the subspace enables us to provide insights for the
improvement of cross-subject generalization with EMG signals.
( 2
min )
Image-based Data Representations of Time Series: A Comparative Analysis in EEG Artifact Detection. (arXiv:2401.05409v1 [eess.SP])
Alternative data representations are powerful tools that augment the
performance of downstream models. However, there is an abundance of such
representations within the machine learning toolbox, and the field lacks a
comparative understanding of the suitability of each representation method.
In this paper, we propose artifact detection and classification within EEG
data as a testbed for profiling image-based data representations of time series
data. We then evaluate eleven popular deep learning architectures on each of
six commonly-used representation methods.
We find that, while the choice of representation entails a choice within the
tradeoff between bias and variance, certain representations are practically
more effective in highlighting features which increase the signal-to-noise
ratio of the data. We present our results on EEG data, and open-source our
testing framework to enable future comparative analyses in this vein.
( 2
min )
An adaptive network-based approach for advanced forecasting of cryptocurrency values. (arXiv:2401.05441v1 [q-fin.ST])
This paper describes an architecture for predicting the price of
cryptocurrencies for the next seven days using the Adaptive Network Based Fuzzy
Inference System (ANFIS). Historical data of cryptocurrencies and indexes that
are considered are Bitcoin (BTC), Ethereum (ETH), Bitcoin Dominance (BTC.D),
and Ethereum Dominance (ETH.D) in a daily timeframe. The methods used to teach
the data are hybrid and backpropagation algorithms, as well as grid partition,
subtractive clustering, and Fuzzy C-means clustering (FCM) algorithms, which
are used in data clustering. The architectural performance designed in this
paper has been compared with different inputs and neural network models in
terms of statistical evaluation criteria. Finally, the proposed method can
predict the price of digital currencies in a short time.
( 2
min )
Hyperspectral Lightcurve Inversion for Attitude Determination. (arXiv:2401.05397v1 [eess.SP])
Spectral lightcurves consisting of time series single-pixel spectral
measurements of spacecraft are used to infer the spacecraft's attitude and
rotation. Two methods are used. One based on numerical optimisation of a
regularised least squares cost function, and another based on machine learning
with a neural network model. The aim is to work with minimal information, thus
no prior is available on the attitude nor on the inertia tensor. The
theoretical and practical aspects of this task are investigated, and the
methodology is tested on synthetic data. Results are shown based on synthetic
data.
( 2
min )
Stable Online and Offline Reinforcement Learning for Antibody CDRH3 Design. (arXiv:2401.05341v1 [q-bio.BM])
The field of antibody-based therapeutics has grown significantly in recent
years, with targeted antibodies emerging as a potentially effective approach to
personalized therapies. Such therapies could be particularly beneficial for
complex, highly individual diseases such as cancer. However, progress in this
field is often constrained by the extensive search space of amino acid
sequences that form the foundation of antibody design. In this study, we
introduce a novel reinforcement learning method specifically tailored to
address the unique challenges of this domain. We demonstrate that our method
can learn the design of high-affinity antibodies against multiple targets in
silico, utilizing either online interaction or offline datasets. To the best of
our knowledge, our approach is the first of its kind and outperforms existing
methods on all tested antigens in the Absolut! database.
( 2
min )
Online Action Recognition for Human Risk Prediction with Anticipated Haptic Alert via Wearables. (arXiv:2401.05365v1 [eess.SP])
This paper proposes a framework that combines online human state estimation,
action recognition and motion prediction to enable early assessment and
prevention of worker biomechanical risk during lifting tasks. The framework
leverages the NIOSH index to perform online risk assessment, thus fitting
real-time applications. In particular, the human state is retrieved via inverse
kinematics/dynamics algorithms from wearable sensor data. Human action
recognition and motion prediction are achieved by implementing an LSTM-based
Guided Mixture of Experts architecture, which is trained offline and inferred
online. With the recognized actions, a single lifting activity is divided into
a series of continuous movements and the Revised NIOSH Lifting Equation can be
applied for risk assessment. Moreover, the predicted motions enable
anticipation of future risks. A haptic actuator, embedded in the wearable
system, can alert the subject of potential risk, acting as an active prevention
device. The performance of the proposed framework is validated by executing
real lifting tasks, while the subject is equipped with the iFeel wearable
system.
( 2
min )
Navigating Privacy and Copyright Challenges Across the Data Lifecycle of Generative AI. (arXiv:2311.18252v2 [cs.SE] UPDATED)
The advent of Generative AI has marked a significant milestone in artificial
intelligence, demonstrating remarkable capabilities in generating realistic
images, texts, and data patterns. However, these advancements come with
heightened concerns over data privacy and copyright infringement, primarily due
to the reliance on vast datasets for model training. Traditional approaches
like differential privacy, machine unlearning, and data poisoning only offer
fragmented solutions to these complex issues. Our paper delves into the
multifaceted challenges of privacy and copyright protection within the data
lifecycle. We advocate for integrated approaches that combines technical
innovation with ethical foresight, holistically addressing these concerns by
investigating and devising solutions that are informed by the lifecycle
perspective. This work aims to catalyze a broader discussion and inspire
concerted efforts towards data privacy and copyright integrity in Generative
AI.
( 2
min )
Transparency in Sleep Staging: Deep Learning Method for EEG Sleep Stage Classification with Model Interpretability. (arXiv:2309.07156v3 [eess.SP] UPDATED)
Automated Sleep stage classification using raw single channel EEG is a
critical tool for sleep quality assessment and disorder diagnosis. However,
modelling the complexity and variability inherent in this signal is a
challenging task, limiting their practicality and effectiveness in clinical
settings. To mitigate these challenges, this study presents an end-to-end deep
learning (DL) model which integrates squeeze and excitation blocks within the
residual network to extract features and stacked Bi-LSTM to understand complex
temporal dependencies. A distinctive aspect of this study is the adaptation of
GradCam for sleep staging, marking the first instance of an explainable DL
model in this domain with alignment of its decision-making with sleep expert's
insights. We evaluated our model on the publically available datasets
(SleepEDF-20, SleepEDF-78, and SHHS), achieving Macro-F1 scores of 82.5, 78.9,
and 81.9, respectively. Additionally, a novel training efficiency enhancement
strategy was implemented by increasing stride size, leading to 8x faster
training times with minimal impact on performance. Comparative analyses
underscore our model outperforms all existing baselines, indicating its
potential for clinical usage.
( 3
min )
Amplitude-Independent Machine Learning for PPG through Visibility Graphs and Transfer Learning. (arXiv:2305.14062v3 [eess.SP] UPDATED)
Photoplethysmography (PPG) refers to the measurement of variations in blood
volume using light and is a feature of most wearable devices. The PPG signals
provide insight into the body's circulatory system and can be employed to
extract various bio-features, such as heart rate and vascular ageing. Although
several algorithms have been proposed for this purpose, many exhibit
limitations, including heavy reliance on human calibration, high signal quality
requirements, and a lack of generalisation. In this paper, we introduce a PPG
signal processing framework that integrates graph theory and computer vision
algorithms, to provide an analysis framework which is amplitude-independent and
invariant to affine transformations. It also requires minimal preprocessing,
fuses information through RGB channels and exhibits robust generalisation
across tasks and datasets. The proposed VGTL-net achieves state-of-the-art
performance in the prediction of vascular ageing and demonstrates robust
estimation of continuous blood pressure waveforms.
( 2
min )
Efficient and Effective Methods for Mixed Precision Neural Network Quantization for Faster, Energy-efficient Inference. (arXiv:2301.13330v2 [cs.LG] UPDATED)
For efficient neural network inference, it is desirable to achieve
state-of-the-art accuracy with the simplest networks requiring the least
computation, memory, and power. Quantizing networks to lower precision is a
powerful technique for simplifying networks. As each layer of a network may
have different sensitivity to quantization, mixed precision quantization
methods selectively tune the precision of individual layers to achieve a
minimum drop in task performance (e.g., accuracy). To estimate the impact of
layer precision choice on task performance, two methods are introduced: i)
Entropy Approximation Guided Layer selection (EAGL) is fast and uses the
entropy of the weight distribution, and ii) Accuracy-aware Layer Precision
Selection (ALPS) is straightforward and relies on single epoch fine-tuning
after layer precision reduction. Using EAGL and ALPS for layer precision
selection, full-precision accuracy is recovered with a mix of 4-bit and 2-bit
layers for ResNet-50, ResNet-101 and BERT-base transformer networks,
demonstrating enhanced performance across the entire accuracy-throughput
frontier. The techniques demonstrate better performance than existing
techniques in several commensurate comparisons. Notably, this is accomplished
with significantly lesser computational time required to reach a solution.
( 2
min )
An unfolding method based on conditional Invertible Neural Networks (cINN) using iterative training. (arXiv:2212.08674v3 [hep-ph] UPDATED)
The unfolding of detector effects is crucial for the comparison of data to
theory predictions. While traditional methods are limited to representing the
data in a low number of dimensions, machine learning has enabled new unfolding
techniques while retaining the full dimensionality. Generative networks like
invertible neural networks~(INN) enable a probabilistic unfolding, which map
individual events to their corresponding unfolded probability distribution. The
accuracy of such methods is however limited by how well simulated training
samples model the actual data that is unfolded. We introduce the iterative
conditional INN~(IcINN) for unfolding that adjusts for deviations between
simulated training samples and data. The IcINN unfolding is first validated on
toy data and then applied to pseudo-data for the $pp \to Z \gamma \gamma$
process.
( 2
min )
CP-PINNs: Changepoints Detection in PDEs using Physics Informed Neural Networks with Total-Variation Penalty. (arXiv:2208.08626v2 [stat.ML] UPDATED)
The paper shows that Physics-Informed Neural Networks (PINNs) can fail to
estimate the correct Partial Differential Equations (PDEs) dynamics in cases of
unknown changepoints in the parameters. To address this, we propose a new
CP-PINNs model which integrates PINNs with Total-Variation penalty for accurate
changepoints detection and PDEs discovery. In order to optimally combine the
tasks of model fitting, PDEs discovery, and changepoints detection, we develop
a new meta-learning algorithm that exploits batch learning to dynamically
refines the optimization objective when moving over the consecutive batches of
the data. Empirically, in case of changepoints in the dynamics, our approach
demonstrates accurate parameter estimation and model alignment, and in case of
no changepoints in the data, it converges numerically to the solution from the
original PINNs model.
( 2
min )
Improving deep neural network generalization and robustness to background bias via layer-wise relevance propagation optimization. (arXiv:2202.00232v7 [eess.IV] UPDATED)
Features in images' backgrounds can spuriously correlate with the images'
classes, representing background bias. They can influence the classifier's
decisions, causing shortcut learning (Clever Hans effect). The phenomenon
generates deep neural networks (DNNs) that perform well on standard evaluation
datasets but generalize poorly to real-world data. Layer-wise Relevance
Propagation (LRP) explains DNNs' decisions. Here, we show that the optimization
of LRP heatmaps can minimize the background bias influence on deep classifiers,
hindering shortcut learning. By not increasing run-time computational cost, the
approach is light and fast. Furthermore, it applies to virtually any
classification architecture. After injecting synthetic bias in images'
backgrounds, we compared our approach (dubbed ISNet) to eight state-of-the-art
DNNs, quantitatively demonstrating its superior robustness to background bias.
Mixed datasets are common for COVID-19 and tuberculosis classification with
chest X-rays, fostering background bias. By focusing on the lungs, the ISNet
reduced shortcut learning. Thus, its generalization performance on external
(out-of-distribution) test databases significantly surpassed all implemented
benchmark models.
( 3
min )
Implications of Noise in Resistive Memory on Deep Neural Networks for Image Classification. (arXiv:2401.05820v1 [cs.LG])
Resistive memory is a promising alternative to SRAM, but is also an
inherently unstable device that requires substantial effort to ensure correct
read and write operations. To avoid the associated costs in terms of area, time
and energy, the present work is concerned with exploring how much noise in
memory operations can be tolerated by image classification tasks based on
neural networks. We introduce a special noisy operator that mimics the noise in
an exemplary resistive memory unit, explore the resilience of convolutional
neural networks on the CIFAR-10 classification task, and discuss a couple of
countermeasures to improve this resilience.
( 2
min )
Cheetah: Bridging the Gap Between Machine Learning and Particle Accelerator Physics with High-Speed, Differentiable Simulations. (arXiv:2401.05815v1 [physics.acc-ph])
Machine learning has emerged as a powerful solution to the modern challenges
in accelerator physics. However, the limited availability of beam time, the
computational cost of simulations, and the high-dimensionality of optimisation
problems pose significant challenges in generating the required data for
training state-of-the-art machine learning models. In this work, we introduce
Cheetah, a PyTorch-based high-speed differentiable linear-beam dynamics code.
Cheetah enables the fast collection of large data sets by reducing computation
times by multiple orders of magnitude and facilitates efficient gradient-based
optimisation for accelerator tuning and system identification. This positions
Cheetah as a user-friendly, readily extensible tool that integrates seamlessly
with widely adopted machine learning tools. We showcase the utility of Cheetah
through five examples, including reinforcement learning training,
gradient-based beamline tuning, gradient-based system identification,
physics-informed Bayesian optimisation priors, and modular neural network
surrogate modelling of space charge effects. The use of such a high-speed
differentiable simulation code will simplify the development of machine
learning-based methods for particle accelerators and fast-track their
integration into everyday operations of accelerator facilities.
( 2
min )
An experimental evaluation of Deep Reinforcement Learning algorithms for HVAC control. (arXiv:2401.05737v1 [cs.LG])
Heating, Ventilation, and Air Conditioning (HVAC) systems are a major driver
of energy consumption in commercial and residential buildings. Recent studies
have shown that Deep Reinforcement Learning (DRL) algorithms can outperform
traditional reactive controllers. However, DRL-based solutions are generally
designed for ad hoc setups and lack standardization for comparison. To fill
this gap, this paper provides a critical and reproducible evaluation, in terms
of comfort and energy consumption, of several state-of-the-art DRL algorithms
for HVAC control. The study examines the controllers' robustness, adaptability,
and trade-off between optimization goals by using the Sinergym framework. The
results obtained confirm the potential of DRL algorithms, such as SAC and TD3,
in complex scenarios and reveal several challenges related to generalization
and incremental learning.
( 2
min )
Segment Boundary Detection via Class Entropy Measurements in Connectionist Phoneme Recognition. (arXiv:2401.05717v1 [eess.AS])
This article investigates the possibility to use the class entropy of the
output of a connectionist phoneme recogniser to predict time boundaries between
phonetic classes. The rationale is that the value of the entropy should
increase in proximity of a transition between two segments that are well
modelled (known) by the recognition network since it is a measure of
uncertainty. The advantage of this measure is its simplicity as the posterior
probabilities of each class are available in connectionist phoneme recognition.
The entropy and a number of measures based on differentiation of the entropy
are used in isolation and in combination. The decision methods for predicting
the boundaries range from simple thresholds to neural network based procedure.
The different methods are compared with respect to their precision, measured in
terms of the ratio between the number C of predicted boundaries within 10 or 20
msec of the reference and the total number of predicted boundaries, and recall,
measured as the ratio between C and the total number of reference boundaries.
( 2
min )
Kernelized Normalizing Constant Estimation: Bridging Bayesian Quadrature and Bayesian Optimization. (arXiv:2401.05716v1 [cs.LG])
In this paper, we study the problem of estimating the normalizing constant
$\int e^{-\lambda f(x)}dx$ through queries to the black-box function $f$, where
$f$ belongs to a reproducing kernel Hilbert space (RKHS), and $\lambda$ is a
problem parameter. We show that to estimate the normalizing constant within a
small relative error, the level of difficulty depends on the value of
$\lambda$: When $\lambda$ approaches zero, the problem is similar to Bayesian
quadrature (BQ), while when $\lambda$ approaches infinity, the problem is
similar to Bayesian optimization (BO). More generally, the problem varies
between BQ and BO. We find that this pattern holds true even when the function
evaluations are noisy, bringing new aspects to this topic. Our findings are
supported by both algorithm-independent lower bounds and algorithmic upper
bounds, as well as simulation studies conducted on a variety of benchmark
functions.
( 2
min )
The Distributional Reward Critic Architecture for Perturbed-Reward Reinforcement Learning. (arXiv:2401.05710v1 [cs.LG])
We study reinforcement learning in the presence of an unknown reward
perturbation. Existing methodologies for this problem make strong assumptions
including reward smoothness, known perturbations, and/or perturbations that do
not modify the optimal policy. We study the case of unknown arbitrary
perturbations that discretize and shuffle reward space, but have the property
that the true reward belongs to the most frequently observed class after
perturbation. This class of perturbations generalizes existing classes (and, in
the limit, all continuous bounded perturbations) and defeats existing methods.
We introduce an adaptive distributional reward critic and show theoretically
that it can recover the true rewards under technical conditions. Under the
targeted perturbation in discrete and continuous control tasks, we win/tie the
highest return in 40/57 settings (compared to 16/57 for the best baseline).
Even under the untargeted perturbation, we still win an edge over the baseline
designed especially for that setting.
( 2
min )
Learning Performance-Oriented Control Barrier Functions Under Complex Safety Constraints and Limited Actuation. (arXiv:2401.05629v1 [cs.LG])
Control Barrier Functions (CBFs) provide an elegant framework for designing
safety filters for nonlinear control systems by constraining their trajectories
to an invariant subset of a prespecified safe set. However, the task of finding
a CBF that concurrently maximizes the volume of the resulting control invariant
set while accommodating complex safety constraints, particularly in high
relative degree systems with actuation constraints, continues to pose a
substantial challenge. In this work, we propose a novel self-supervised
learning framework that holistically addresses these hurdles. Given a Boolean
composition of multiple state constraints that define the safe set, our
approach starts with building a single continuously differentiable function
whose 0-superlevel set provides an inner approximation of the safe set. We then
use this function together with a smooth neural network to parameterize the CBF
candidate. Finally, we design a training loss function based on a
Hamilton-Jacobi partial differential equation to train the CBF while enlarging
the volume of the induced control invariant set. We demonstrate the
effectiveness of our approach via numerical experiments.
( 2
min )
VI-PANN: Harnessing Transfer Learning and Uncertainty-Aware Variational Inference for Improved Generalization in Audio Pattern Recognition. (arXiv:2401.05531v1 [cs.LG])
Transfer learning (TL) is an increasingly popular approach to training deep
learning (DL) models that leverages the knowledge gained by training a
foundation model on diverse, large-scale datasets for use on downstream tasks
where less domain- or task-specific data is available. The literature is rich
with TL techniques and applications; however, the bulk of the research makes
use of deterministic DL models which are often uncalibrated and lack the
ability to communicate a measure of epistemic (model) uncertainty in
prediction. Unlike their deterministic counterparts, Bayesian DL (BDL) models
are often well-calibrated, provide access to epistemic uncertainty for a
prediction, and are capable of achieving competitive predictive performance. In
this study, we propose variational inference pre-trained audio neural networks
(VI-PANNs). VI-PANNs are a variational inference variant of the popular
ResNet-54 architecture which are pre-trained on AudioSet, a large-scale audio
event detection dataset. We evaluate the quality of the resulting uncertainty
when transferring knowledge from VI-PANNs to other downstream acoustic
classification tasks using the ESC-50, UrbanSound8K, and DCASE2013 datasets. We
demonstrate, for the first time, that it is possible to transfer calibrated
uncertainty information along with knowledge from upstream tasks to enhance a
model's capability to perform downstream tasks.
( 2
min )
The recursive scheme of clustering. (arXiv:2401.05479v1 [cs.LG])
The problem of data clustering is one of the most important in data analysis.
It can be problematic when dealing with experimental data characterized by
measurement uncertainties and errors. Our paper proposes a recursive scheme for
clustering data obtained in geographical (climatological) experiments. The
discussion of results obtained by k-means and SOM methods with the developed
recursive procedure is presented. We show that the clustering using the new
approach gives more acceptable results when compared to experts assessments.
( 2
min )
Population Graph Cross-Network Node Classification for Autism Detection Across Sample Groups. (arXiv:2401.05478v1 [cs.SI])
Graph neural networks (GNN) are a powerful tool for combining imaging and
non-imaging medical information for node classification tasks. Cross-network
node classification extends GNN techniques to account for domain drift,
allowing for node classification on an unlabeled target network. In this paper
we present OTGCN, a powerful, novel approach to cross-network node
classification. This approach leans on concepts from graph convolutional
networks to harness insights from graph data structures while simultaneously
applying strategies rooted in optimal transport to correct for the domain drift
that can occur between samples from different data collection sites. This
blended approach provides a practical solution for scenarios with many distinct
forms of data collected across different locations and equipment. We
demonstrate the effectiveness of this approach at classifying Autism Spectrum
Disorder subjects using a blend of imaging and non-imaging data.
( 2
min )
Introducing New Node Prediction in Graph Mining: Predicting All Links from Isolated Nodes with Graph Neural Networks. (arXiv:2401.05468v1 [cs.SI])
This paper introduces a new problem in the field of graph mining and social
network analysis called new node prediction. More technically, the task can be
categorized as zero-shot out-of-graph all-links prediction. This challenging
problem aims to predict all links from a new, isolated, and unobserved node
that was previously disconnected from the graph. Unlike classic approaches to
link prediction (including few-shot out-of-graph link prediction), this problem
presents two key differences: (1) the new node has no existing links from which
to extract patterns for new predictions; and (2) the goal is to predict not
just one, but all the links of this new node, or at least a significant part of
them. Experiments demonstrate that an architecture based on Deep Graph Neural
Networks can learn to solve this challenging problem in a bibliographic
citation network.
( 2
min )
Dimensionality-Aware Outlier Detection: Theoretical and Experimental Analysis. (arXiv:2401.05453v1 [cs.LG])
We present a nonparametric method for outlier detection that takes full
account of local variations in intrinsic dimensionality within the dataset.
Using the theory of Local Intrinsic Dimensionality (LID), our
'dimensionality-aware' outlier detection method, DAO, is derived as an
estimator of an asymptotic local expected density ratio involving the query
point and a close neighbor drawn at random. The dimensionality-aware behavior
of DAO is due to its use of local estimation of LID values in a
theoretically-justified way. Through comprehensive experimentation on more than
800 synthetic and real datasets, we show that DAO significantly outperforms
three popular and important benchmark outlier detection methods: Local Outlier
Factor (LOF), Simplified LOF, and kNN.
( 2
min )
Cuff-less Arterial Blood Pressure Waveform Synthesis from Single-site PPG using Transformer & Frequency-domain Learning. (arXiv:2401.05452v1 [eess.SP])
We propose two novel purpose-built deep learning (DL) models for synthesis of
the arterial blood pressure (ABP) waveform in a cuff-less manner, using a
single-site photoplethysmography (PPG) signal. We utilize the public UCI
dataset on cuff-less blood pressure (CLBP) estimation to train and evaluate our
DL models. Firstly, we implement a transformer model that incorporates
positional encoding, multi-head attention, layer normalization, and dropout
techniques, and synthesizes the ABP waveform with a mean absolute error (MAE)
of 14. Secondly, we implement a frequency-domain (FD) learning approach where
we first obtain the discrete cosine transform (DCT) coefficients of the PPG and
ABP signals corresponding to two cardiac cycles, and then learn a
linear/non-linear (L/NL) regression between them. We learn that the FD L/NL
regression model outperforms the transformer model by achieving an MAE of 11.87
and 8.01, for diastolic blood pressure (DBP) and systolic blood pressure (SBP),
respectively. Our FD L/NL regression model also fulfills the AAMI criterion of
utilizing data from more than 85 subjects, and achieves grade B by the BHS
criterion.
( 2
min )
ANALYTiC: Understanding Decision Boundaries and Dimensionality Reduction in Machine Learning. (arXiv:2401.05418v1 [eess.SP])
The advent of compact, handheld devices has given us a pool of tracked
movement data that could be used to infer trends and patterns that can be made
to use. With this flooding of various trajectory data of animals, humans,
vehicles, etc., the idea of ANALYTiC originated, using active learning to infer
semantic annotations from the trajectories by learning from sets of labeled
data. This study explores the application of dimensionality reduction and
decision boundaries in combination with the already present active learning,
highlighting patterns and clusters in data. We test these features with three
different trajectory datasets with objective of exploiting the the already
labeled data and enhance their interpretability. Our experimental analysis
exemplifies the potential of these combined methodologies in improving the
efficiency and accuracy of trajectory labeling. This study serves as a
stepping-stone towards the broader integration of machine learning and visual
methods in context of movement data analysis.
( 2
min )
On the Three Demons in Causality in Finance: Time Resolution, Nonstationarity, and Latent Factors. (arXiv:2401.05414v1 [q-fin.ST])
Financial data is generally time series in essence and thus suffers from
three fundamental issues: the mismatch in time resolution, the time-varying
property of the distribution - nonstationarity, and causal factors that are
important but unknown/unobserved. In this paper, we follow a causal perspective
to systematically look into these three demons in finance. Specifically, we
reexamine these issues in the context of causality, which gives rise to a novel
and inspiring understanding of how the issues can be addressed. Following this
perspective, we provide systematic solutions to these problems, which hopefully
would serve as a foundation for future research in the area.
( 2
min )
Bayesian ECG reconstruction using denoising diffusion generative models. (arXiv:2401.05388v1 [eess.SP])
In this work, we propose a denoising diffusion generative model (DDGM)
trained with healthy electrocardiogram (ECG) data that focuses on ECG
morphology and inter-lead dependence. Our results show that this innovative
generative model can successfully generate realistic ECG signals. Furthermore,
we explore the application of recent breakthroughs in solving linear inverse
Bayesian problems using DDGM. This approach enables the development of several
important clinical tools. These include the calculation of corrected QT
intervals (QTc), effective noise suppression of ECG signals, recovery of
missing ECG leads, and identification of anomalous readings, enabling
significant advances in cardiac health monitoring and diagnosis.
( 2
min )
Angle-Equivariant Convolutional Neural Networks for Interference Mitigation in Automotive Radar. (arXiv:2401.05385v1 [eess.SP])
In automotive applications, frequency modulated continuous wave (FMCW) radar
is an established technology to determine the distance, velocity and angle of
objects in the vicinity of the vehicle. The quality of predictions might be
seriously impaired if mutual interference between radar sensors occurs.
Previous work processes data from the entire receiver array in parallel to
increase interference mitigation quality using neural networks (NNs). However,
these architectures do not generalize well across different angles of arrival
(AoAs) of interferences and objects. In this paper we introduce fully
convolutional neural network (CNN) with rank-three convolutions which is able
to transfer learned patterns between different AoAs. Our proposed architecture
outperforms previous work while having higher robustness and a lower number of
trainable parameters. We evaluate our network on a diverse data set and
demonstrate its angle equivariance.
( 2
min )
Rethinking Performance Measures of RNA Secondary Structure Problems. (arXiv:2401.05351v1 [q-bio.BM])
Accurate RNA secondary structure prediction is vital for understanding
cellular regulation and disease mechanisms. Deep learning (DL) methods have
surpassed traditional algorithms by predicting complex features like
pseudoknots and multi-interacting base pairs. However, traditional distance
measures can hardly deal with such tertiary interactions and the currently used
evaluation measures (F1 score, MCC) have limitations. We propose the
Weisfeiler-Lehman graph kernel (WL) as an alternative metric. Embracing
graph-based metrics like WL enables fair and accurate evaluation of RNA
structure prediction algorithms. Further, WL provides informative guidance, as
demonstrated in an RNA design experiment.
( 2
min )
STR-Cert: Robustness Certification for Deep Text Recognition on Deep Learning Pipelines and Vision Transformers. (arXiv:2401.05338v1 [cs.CV])
Robustness certification, which aims to formally certify the predictions of
neural networks against adversarial inputs, has become an integral part of
important tool for safety-critical applications. Despite considerable progress,
existing certification methods are limited to elementary architectures, such as
convolutional networks, recurrent networks and recently Transformers, on
benchmark datasets such as MNIST. In this paper, we focus on the robustness
certification of scene text recognition (STR), which is a complex and
extensively deployed image-based sequence prediction problem. We tackle three
types of STR model architectures, including the standard STR pipelines and the
Vision Transformer. We propose STR-Cert, the first certification method for STR
models, by significantly extending the DeepPoly polyhedral verification
framework via deriving novel polyhedral bounds and algorithms for key STR model
components. Finally, we certify and compare STR models on six datasets,
demonstrating the efficiency and scalability of robustness certification,
particularly for the Vision Transformer.
( 2
min )
Optimal Linear Signal: An Unsupervised Machine Learning Framework to Optimize PnL with Linear Signals. (arXiv:2401.05337v1 [q-fin.ST])
This study presents an unsupervised machine learning approach for optimizing
Profit and Loss (PnL) in quantitative finance. Our algorithm, akin to an
unsupervised variant of linear regression, maximizes the Sharpe Ratio of PnL
generated from signals constructed linearly from exogenous variables. The
methodology employs a linear relationship between exogenous variables and the
trading signal, with the objective of maximizing the Sharpe Ratio through
parameter optimization. Empirical application on an ETF representing U.S.
Treasury bonds demonstrates the model's effectiveness, supported by
regularization techniques to mitigate overfitting. The study concludes with
potential avenues for further development, including generalized time steps and
enhanced corrective terms.
( 2
min )
-
Open
TaCo: Targeted Concept Removal in Output Embeddings for NLP via Information Theory and Explainability. (arXiv:2312.06499v2 [cs.CL] UPDATED)
The fairness of Natural Language Processing (NLP) models has emerged as a
crucial concern. Information theory indicates that to achieve fairness, a model
should not be able to predict sensitive variables, such as gender, ethnicity,
and age. However, information related to these variables often appears
implicitly in language, posing a challenge in identifying and mitigating biases
effectively. To tackle this issue, we present a novel approach that operates at
the embedding level of an NLP model, independent of the specific architecture.
Our method leverages insights from recent advances in XAI techniques and
employs an embedding transformation to eliminate implicit information from a
selected variable. By directly manipulating the embeddings in the final layer,
our approach enables a seamless integration into existing models without
requiring significant modifications or retraining. In evaluation, we show that
the proposed post-hoc approach significantly reduces gender-related
associations in NLP models while preserving the overall performance and
functionality of the models. An implementation of our method is available:
https://github.com/fanny-jourdan/TaCo
( 2
min )
CP-PINNs: Changepoints Detection in PDEs using Physics Informed Neural Networks with Total-Variation Penalty. (arXiv:2208.08626v2 [stat.ML] UPDATED)
The paper shows that Physics-Informed Neural Networks (PINNs) can fail to
estimate the correct Partial Differential Equations (PDEs) dynamics in cases of
unknown changepoints in the parameters. To address this, we propose a new
CP-PINNs model which integrates PINNs with Total-Variation penalty for accurate
changepoints detection and PDEs discovery. In order to optimally combine the
tasks of model fitting, PDEs discovery, and changepoints detection, we develop
a new meta-learning algorithm that exploits batch learning to dynamically
refines the optimization objective when moving over the consecutive batches of
the data. Empirically, in case of changepoints in the dynamics, our approach
demonstrates accurate parameter estimation and model alignment, and in case of
no changepoints in the data, it converges numerically to the solution from the
original PINNs model.
( 2
min )
Kernelized Normalizing Constant Estimation: Bridging Bayesian Quadrature and Bayesian Optimization. (arXiv:2401.05716v1 [cs.LG])
In this paper, we study the problem of estimating the normalizing constant
$\int e^{-\lambda f(x)}dx$ through queries to the black-box function $f$, where
$f$ belongs to a reproducing kernel Hilbert space (RKHS), and $\lambda$ is a
problem parameter. We show that to estimate the normalizing constant within a
small relative error, the level of difficulty depends on the value of
$\lambda$: When $\lambda$ approaches zero, the problem is similar to Bayesian
quadrature (BQ), while when $\lambda$ approaches infinity, the problem is
similar to Bayesian optimization (BO). More generally, the problem varies
between BQ and BO. We find that this pattern holds true even when the function
evaluations are noisy, bringing new aspects to this topic. Our findings are
supported by both algorithm-independent lower bounds and algorithmic upper
bounds, as well as simulation studies conducted on a variety of benchmark
functions.
( 2
min )
Bayesian ECG reconstruction using denoising diffusion generative models. (arXiv:2401.05388v1 [eess.SP])
In this work, we propose a denoising diffusion generative model (DDGM)
trained with healthy electrocardiogram (ECG) data that focuses on ECG
morphology and inter-lead dependence. Our results show that this innovative
generative model can successfully generate realistic ECG signals. Furthermore,
we explore the application of recent breakthroughs in solving linear inverse
Bayesian problems using DDGM. This approach enables the development of several
important clinical tools. These include the calculation of corrected QT
intervals (QTc), effective noise suppression of ECG signals, recovery of
missing ECG leads, and identification of anomalous readings, enabling
significant advances in cardiac health monitoring and diagnosis.
( 2
min )
